Unstructured Environments Research Articles

Exploring the Use of Deep Reinforcement Learning for Autonomous Navigation in Unstructured Environments

Abstract: Autonomous navigation in unstructured environments poses a significant challenge in robotics and artificial intelligence. The capability to navigate through dynamic and unpredictable terrains such as disaster zones, outdoor landscapes, or congested urban settings demands sophisticated solutions. This research paper delves into the imperative role of Deep Reinforcement Learning (DRL) in addressing these challenges and advancing the field of autonomous navigation. The core necessity of this research paper lies in the application and exploration of DRL within the realm of autonomous navigation. By leveraging neural networks and reinforcement learning algorithms, autonomous agents can dynamically navigate through unstructured environments without explicit programming or human intervention. Instead, they learn to navigate by receiving feedback through rewards or penalties, thereby continuously improving their decision-making processes. Through a comprehensive review of existing literature and experiments this paper aims to elucidate the pivotal role of DRL in shaping the future of autonomous navigation. It highlights the necessity of robust and adaptive systems capable of navigating unstructured environments, emphasizing the transformative potential of DRL in revolutionizing autonomous systems' capabilities.

Advanced Flexible Sensing Technologies for Soft Robots

AbstractSoft robots have recently attracted increasing interest due to their advantages in durability, flexibility, and deformability, which enable them to adapt to unstructured environments and perform various complex tasks. Perception is crucial for soft robots. To better mimic biological systems, sensors need to be integrated into soft robotic systems to obtain both proprioceptive and external perception for effective usage. This review summarizes the latest advancements in flexible sensing feedback technologies for soft robotic applications. It begins with an introduction to the development of various flexible sensors for soft robots, followed by an in‐depth exploration of smart materials and advanced manufacturing methods. A detailed description of flexible sensing modalities and methodologies is also included in the review to illustrate the continuous breakthrough of the technology. In addition, the applications of soft robots based on these advanced sensing technologies are concluded as well. The challenges of flexible sensing technologies for soft robots and promising solutions are finally discussed and analyzed to provide a prospect for future development. By examining the recent advances in intelligent flexible sensing technologies, this review is dedicated to highlighting the potential of soft robotics and motivating innovation within the field.

Localization of asparagus spears using time-of-flight imaging for robotic harvesting

Purpose Time of flight (ToF) imaging is a promising emerging technology for the purposes of crop identification. This paper aim to presents localization system for identifying and localizing asparagus in the field based on point clouds from ToF imaging. Since the semantics are not included in the point cloud, it contains the geometric information of other objects such as stones and weeds other than asparagus spears. An approach is required for extracting the spear information so that a robotic system can be used for harvesting. Design/methodology/approach A real-time convolutional neural network (CNN)-based method is used for filtering the point cloud generated by a ToF camera, allowing subsequent processing methods to operate over smaller and more information-dense data sets, resulting in reduced processing time. The segmented point cloud can then be split into clusters of points representing each individual spear. Geometric filters are developed to eliminate the non-asparagus points in each cluster so that each spear can be modelled and localized. The spear information can then be used for harvesting decisions. Findings The localization system is integrated into a robotic harvesting prototype system. Several field trials have been conducted with satisfactory performance. The identification of a spear from the point cloud is the key to successful localization. Segmentation and clustering points into individual spears are two major failures for future improvements. Originality/value Most crop localizations in agricultural robotic applications using ToF imaging technology are implemented in a very controlled environment, such as a greenhouse. The target crop and the robotic system are stationary during the localization process. The novel proposed method for asparagus localization has been tested in outdoor farms and integrated with a robotic harvesting platform. Asparagus detection and localization are achieved in real time on a continuously moving robotic platform in a cluttered and unstructured environment.

A Biohybrid Self-Dispersing Miniature Machine Using Wild Oat Fruit Awns for Reforestation and Precision Agriculture.

Advances in bioinspired and biohybrid robotics are enabling the creation of multifunctional systems able to explore complex unstructured environments. Inspired by Avena fruits, we report a biohybrid miniaturized autonomous machine (HybriBot) composed of a biomimetic biodegradable capsule as cargo delivery system and natural humidity-driven sister awns as biological motors. We employ micro-computed tomography, molding via two-photon polymerization and casting of natural awns into biodegradable materials to fabricate multiple HybriBots capable of exploring various soil and navigating soil irregularities, such as holes and cracks. These machines replicate the dispersal movements and biomechanical performances of natural fruits, achieving comparable capsule drag forces up to ≈0.38 N and awns torque up to ≈100 mN*mm. They were functionalized with fertilizer and have been successfully utilized to germinate selected diaspores. HybriBots function as self-dispersed systems with applications in reforestation and precision agriculture. This article is protected by copyright. All rights reserved.

A cloud-edge computing architecture for monitoring protective equipment

The proper use of protective equipment is very important to avoid fatalities. One sector in which this has a great impact is that of construction sites, where a large number of workers die each year. In this sector as in others, employers are responsible for providing their employees with this equipment. In addition, employers must monitor and ensure its correct use. These tasks are usually performed using manual procedures. Existing tools to automate this process are unreliable and present scalability issues. In this paper, we research the benefits of using a cloud-edge computing architecture to automate the monitoring of protective equipment. The solution we propose successfully addresses all the problems that appear in hostile and unstructured work environments such as construction sites. Although these sites are used as a use case, the approach presented can also be deployed in other sectors with similar characteristics and restrictions.

Active robotic search for victims using ensemble deep learning techniques

In recent years, legged quadruped robots have proved to be a valuable support to humans in dealing with search and rescue operations. These robots can move with great ability in complex terrains, unstructured environments or regions with many obstacles. This work employs the quadruped robot A1 Rescue Tasks UPM Robot (ARTU-R) by Unitree, equipped with an RGB-D camera and a lidar, to perform victim searches in post-disaster scenarios. Exploration is done not by following a pre-planned path (as common methods) but by prioritising the areas most likely to harbour victims. To accomplish that task, both indirect search and next best view techniques have been used. When ARTU-R gets inside an unstructured and unknown environment, it selects the next exploration point from a series of candidates. This operation is performed by comparing, for each candidate, the distance to reach it, the unexplored space around it and the probability of a victim being in its vicinity. This probability value is obtained using a Random Forest, which processes the information provided by a convolutional neural network. Unlike other AI techniques, random forests are not black box models; humans can understand their decision-making processes. The system, once integrated, achieves speeds comparable to other state-of-the-art algorithms in terms of exploration, but concerning victim detection, the tests show that the resulting smart exploration generates logical paths—from a human point of view—and that ARTU-R tends to move first to the regions where victims are present.

Analytical Formalism for Data Representation and Object Detection with 2D LiDAR: Application in Mobile Robotics.

In mobile robotics, LASER scanners have a wide spectrum of indoor and outdoor applications, both in structured and unstructured environments, due to their accuracy and precision. Most works that use this sensor have their own data representation and their own case-specific modeling strategies, and no common formalism is adopted. To address this issue, this manuscript presents an analytical approach for the identification and localization of objects using 2D LiDARs. Our main contribution lies in formally defining LASER sensor measurements and their representation, the identification of objects, their main properties, and their location in a scene. We validate our proposal with experiments in generic semi-structured environments common in autonomous navigation, and we demonstrate its feasibility in multiple object detection and identification, strictly following its analytical representation. Finally, our proposal further encourages and facilitates the design, modeling, and implementation of other applications that use LASER scanners as a distance sensor.

Estimation of electrical muscle activity during gait using inertial measurement units with convolution attention neural network and small-scale dataset

In general, muscle activity can be directly measured using Electromyography (EMG) or calculated with musculoskeletal models. However, both methods are not suitable for non-technical users and unstructured environments. It is desired to establish more portable and easy-to-use muscle activity estimation methods. Deep learning (DL) models combined with inertial measurement units (IMUs) have shown great potential to estimate muscle activity. However, it frequently occurs in clinical scenarios that a very small amount of data is available and leads to limited performance of the DL models, while the augmentation techniques to efficiently expand a small sample size for DL model training are rarely used. The primary aim of the present study was to develop a novel DL model to estimate the EMG envelope during gait using IMUs with high accuracy. A secondary aim was to develop a novel model-based data augmentation method to improve the performance of the estimation model with small-scale dataset. Therefore, in the present study, a time convolutional network-based generative adversarial network, namely MuscleGAN, was proposed for data augmentation. Moreover, a subject-independent regression DL model was developed to estimate EMG envelope. Results suggested that the proposed two-stage method has better generalization and estimation performance than the commonly used existing methods. Pearson correlation coefficient and normalized root-mean-square errors derived from the proposed method reached up to 0.72 and 0.13, respectively. It was indicated that the MuscleGAN indeed improved the estimation accuracy of lower limb EMG envelope from 70% to 72%. Thus, even using only two IMUs and a very small-scale dataset, the proposed model is still capable of accurately estimating lower limb EMG envelope, demonstrating considerable potential for its application in clinical and daily life scenarios.

Efficient Stacking and Grasping in Unstructured Environments

Robotics has been booming in recent years. Especially with the development of artificial intelligence, more and more researchers have devoted themselves to the field of robotics, but there are still many shortcomings in the multi-task operation of robots. Reinforcement learning has achieved good performance in manipulator manipulation, especially in grasping, but grasping is only the first step for the robot to perform actions, and it often ignores the stacking, assembly, placement, and other tasks to be carried out later. Such long-horizon tasks still face the problems of expensive time, dead-end exploration, and process reversal. Hierarchical reinforcement learning has some advantages in solving the above problems, but not all tasks can be learned hierarchically. This paper mainly solves the complex manipulation task of continuous multi-action of the manipulator by improving the method of hierarchical reinforcement learning, aiming to solve the task of long sequences such as stacking and alignment by proposing a framework. Our framework completes simulation experiments on various tasks and improves the success rate from 78.3% to 94.8% when cleaning cluttered toys. In the stacking toy experiment, the training speed is nearly three times faster than the baseline method. And our method can be generalized to other long-horizon tasks. Experiments show that the more complex the task, the greater the advantage of our framework.

Large-scale aerial scene perception based on self-supervised multi-view stereo via cycled generative adversarial network

Unmanned aerial vehicle (UAV) has the characteristics of strong maneuverability and wide field of vision, and its application in real-time terrain perception and target capture is one of the active frontier topics in the field of UAV cooperative situation awareness. LiDAR-based method has unique advantages, but data acquisition is difficult and costly. In comparison, multiple view stereo-based (MVS) perception method contains rich image information with low cost, which is relatively easy to be collected. MVS also has potential application value in human–computer interaction and robot positioning (unstructured environment perception, visual servo). To reduce the dependence on costly ground truth, the current self-supervised MVS works calculate the training loss by assuming that the same space points projected into multiple views will share the same RGB information. However, this gap occurs mainly because in the real scene, interference factors such as specular reflection, illumination, and noise will cause large color changes in the source images, which will make it difficult to restrict the photometric consistency of different perspectives with ideal self-supervised loss. To realize more accurate scene perception, a novel self-supervised MVS integrated with generative adversarial network (GAN) is proposed. The overall framework is stacked twice by inputting real images and synthetic images into the network in turn, and adversarial learning is introduced to jointly distinguish the images. In addition, the consistency loss of predicted depth maps from real images and synthetic images is designed to improve the robustness of feature matching and color anti-noise ability. To the best of our knowledge, the proposed work is the first end-to-end depth inference model in MVS field that introduces the adversial training based on GAN, which is an effective complement to self-supervised scene reconstruction. Comprehensive experiment results on the public datasets show that the proposed work is comparable to the mainstream self-supervised MVS.

Human-Following Strategy for Orchard Mobile Robot Based on the KCF-YOLO Algorithm

Autonomous mobile robots play a vital role in the mechanized production of orchards, where human-following is a crucial collaborative function. In unstructured orchard environments, obstacles often obscure the path, and personnel may overlap, leading to significant disruptions to human-following. This paper introduces the KCF-YOLO fusion visual tracking method to ensure stable tracking in interference environments. The YOLO algorithm provides the main framework, and the KCF algorithm intervenes in assistant tracking. A three-dimensional binocular-vision reconstruction method was used to acquire personnel positions, achieving stabilized visual tracking in disturbed environments. The robot was guided by fitting the personnel’s trajectory using an unscented Kalman filter algorithm. The experimental results show that, with 30 trials in multi-person scenarios, the average tracking success rate is 96.66%, with an average frame rate of 8 FPS. Additionally, the mobile robot is capable of maintaining a stable following speed with the target individuals. Across three human-following experiments, the horizontal offset Error Y does not exceed 1.03 m. The proposed KCF-YOLO tracking method significantly bolsters the stability and robustness of the mobile robot for human-following in intricate orchard scenarios, offering an effective solution for tracking tasks.

Bayesian Strategy Networks Based Soft Actor-Critic Learning

A strategy refers to the rules that the agent chooses the available actions to achieve goals. Adopting reasonable strategies is challenging but crucial for an intelligent agent with limited resources working in hazardous, unstructured, and dynamic environments to improve the system’s utility, decrease the overall cost, and increase mission success probability. This article proposes a novel hierarchical strategy decomposition approach based on Bayesian chaining to separate an intricate policy into several simple sub-policies and organize their relationships as Bayesian strategy networks (BSN). We integrate this approach into the state-of-the-art DRL method—soft actor-critic (SAC), and build the corresponding Bayesian soft actor-critic (BSAC) model by organizing several sub-policies as a joint policy. Our method achieves the state-of-the-art performance on the standard continuous control benchmarks in the OpenAI Gym environment. The results demonstrate that the promising potential of the BSAC method significantly improves training efficiency. Furthermore, we extend the topic to the Multi-Agent systems (MAS), discussing the potential research fields and directions.

High-fidelity 3D reconstruction of plants using Neural Radiance Fields

Accurate reconstruction of plant phenotypes plays a key role in optimizing sustainable farming practices in the field of Precision Agriculture (PA). Currently, optical sensor-based approaches dominate the field, but the need for high-fidelity 3D reconstruction of crops and plants in unstructured agricultural environments remains challenging. Recently, a promising development has emerged in the form of Neural Radiance Fields (NeRF), a novel method that utilizes neural density fields. This technology has shown impressive performance in various novel vision synthesis tasks, but has remained relatively unexplored in the agricultural context. In our study, we focus on two fundamental tasks within plant phenotyping: (1) the synthesis of 2D novel-view images and (2) the 3D reconstruction of crop and plant models. We explore the world of NeRF, in particular two state-of-the-art (SOTA) methods: Instant-NGP, which excels in generating high-quality images with impressive training and inference speed, and Instant-NSR, which improves the reconstructed geometry by incorporating the Signed Distance Function (SDF) during training. In particular, we present a novel plant phenotype dataset comprising real plant images from production environments. This dataset is a first-of-its-kind initiative aimed at comprehensively exploring the advantages and limitations of NeRF in agricultural contexts. Our experimental results show that NeRF demonstrates commendable performance in the synthesis of novel-view images and is able to achieve reconstruction results that are competitive with Reality Capture, a leading commercial software for 3D Multi-View Stereo (MVS)-based reconstruction. Moreover, our study also highlights certain drawbacks of NeRF, including relatively slow training speeds, performance limitations in cases of insufficient sampling, and challenges in obtaining geometry quality in complex setups. In conclusion, NeRF introduces a new paradigm in plant phenotyping, providing a powerful tool capable of generating multiple representations, such as multi-view images, point cloud and mesh, from a single process.

A novel method for underactuated UUV tracking unknown contour based on forward-looking sonar

Tracking unknown contour is an essential problem for automated operations of Unmanned Underwater Vehicle (UUV). Applications such as harbor surveillance, island survey, and obstacle avoidance require UUV with the ability to track the contours of unknown marine structures. The absence of a priori information poses a great challenge in solving such problems. Most existing approaches follow the sense-act pattern by designing elaborate control laws to solve the problems under specific configurations. However, they demand much complexity to implement and lack generalizability in unstructured environments. This paper presents a novel unknown contour tracking method based on the sense-plan-act formulation and focuses primarily on the first two processes to make results applicable to conventional controllers. In terms of perception, forward-looking sonar (FLS) is used as the main device of data collection and a probabilistic perception model and a rolling occupancy grid map construction method based on this model are introduced. For planning, it is assumed that the UUV is underactuated and maneuvers in a 2D plane. Considering multiple tracking metrics, the problem is described as a multi-objective optimization problem and an emerging swarm intelligence algorithm is chosen as the solver. Notably, a contour prediction mechanism and a real-time risk assessment and replanning strategy are designed to ensure the continuity and navigational safety of the tracking process, respectively. Results demonstrate the effectiveness of the proposed method with contours extracted from real harbor and island.

Bioinspired Flexible Capacitive Sensor for Robot Positioning in Unstructured Environments.

The evolution of bionic machines into intelligent robots to adapt to real scenarios is inseparable from positioning sensors. However, traditional positioning methods such as camera arrays, ultrasound, or GPS are limited in narrow concealed spaces, harsh temperatures, or dynamic light fields, which hinder the practical application of special robots. Here, we report a flexible sensor inspired by Gnathonemus petersii that enables robots to achieve contactless and high-precision spatial localization independent of the unstructured features of the environment. Sensors are obtained from low-cost materials (carbon nanotubes and polyimides) and simple structures (fibers) and preparation processes (spin-coating). Experiments and simulations confirmed the high resolution (<1 mm) of the sensor over a large distance detection range (>150 mm) and high bandwidth (0-520 MPa) of contact force. Moreover, the sensing capability is still feasible when the sensor is bent to various curvatures and not affected under harsh conditions such as ultralow temperatures (below -78 °C), ultrahigh temperatures (over 250 °C), darkness, or brightness. We demonstrate the practical potential of the proposed sensors for a biomimetic hyper-redundant continuum robot to locate and avoid collisions in unstructured environments.

Determination of garlic clove orientation based on capacitive sensing technology

Garlic is a versatile crop of high economic value, with an increase in the growth of the cultivation scale. Planting garlic with bottom-side (root) down and pointy-tip (clove) up is essential in agriculture because the orientation of the clove significantly affects garlic quality. In this study, we propose a clove orientation recognition technique based on capacitance sensing technology, where clove states are determined by utilizing characteristic differences in capacitance variations associated with different orientations of cloves. First, we applied Maxwell simulations to obtain capacitance variation during the falling process. Second, we conducted field experiments using the capacitive sensing device to obtain the capacitance variations during the falling of garlic in an unstructured environment. Third, the continuous capacitance signal collected during the field experiment was segmented into short-term feature signals containing individual garlic fallings. Finally, Long Short-Term Memory (LSTM) and Frequency-Residual-GoogLeNet (F-Res-GoogLeNet) deep models were trained to recognize garlic falling states. Our best model achieved an accuracy of 96.75%, which meets the agricultural requirements for garlic cultivation. This study demonstrates that monitoring capacitance data in field environments can identify garlic cloves’ orientation, eventually enhancing garlic’s final yield, quality, and economic benefits.

Investigating Suitable Combinations of Dynamic Models and Control Techniques for Offline Reinforcement Learning Based Navigation: Application of Universal Omni-Wheeled Robots

Abstract Omnidirectional locomotion provides wheeled mobile robots (WMR) with better maneuverability and flexibility, which enhances their energy efficiency and dexterity. Universal omni-wheels are one of the best categories of wheels that can be used to develop a WMR (Amarasiri et al., 2022, “Robust Dynamic Modeling and Trajectory Tracking Controller of a Universal Omni-Wheeled Mobile Robot,” ASME Letters Dyn. Sys. Control., 2(4), p. 040902. 10.1115/1.4055690). We study dynamic modeling and controllers for mobile robots to train in a reinforcement learning (RL)-based navigation algorithm. RL tasks require copious amounts of learning iteration episodes, which makes training very time consuming. The choice of dynamic model and controller has a significant impact on training time. In this paper, we compare a traditional Kane’s equations model to a non-holonomic canonical momenta model (Barhorst, 2019, “Generalized Momenta in Constrained Non-Holonomic Systems—Another Perspective on the Canonical Equations of Motion,” Int. J. Non-Linear Mech., 113, pp. 128–145.). We implemented four controllers: proportional integral derivative, linear quadratic regulator with integral action, pole placement, and a full nonlinear sliding mode controller. We summarize the pros and cons of each of the modeling techniques, and control laws implemented. The outcomes of our analysis will improve RL training time for path generation in unstructured environments.

Traversable map construction and robust localization for unstructured road environments1

Intelligent vehicles require accurate identification of traversable road areas and the ability to provide precise and real-time localization data in unstructured road environments. To address these issues, we propose a system for traversable map construction and robust localization in unstructured road environments based on a priori knowledge. The proposed method performs traversable area segmentation on the LiDAR point cloud and employs a submap strategy to jointly optimize multiple frames of data to obtain a reliable and accurate point cloud map of the traversable area, which is then rasterized and combined with the vehicle kinematic model for global path planning. Then, it integrates priori map information and real-time sensor information to provide confidence and priori constraints to ensure the robustness of localization, and it fuses multi-sensor heterogeneous data to improve real-time localization. Experiments are conducted in a mining environment to evaluate the performance of the proposed method on an unstructured road. The experimental results demonstrate that the traversable map and localization results based on the proposed method can meet the requirements for autonomous vehicle driving on unstructured roads and provide reliable priori foundation and localization information for autonomous vehicle navigation.

T-YOLO: a lightweight and efficient detection model for nutrient buds in complex tea-plantation environments.

Quick and accurate detection of nutrient buds is essential for yield prediction and field management in tea plantations. However, the complexity of tea plantation environments and the similarity in color between nutrient buds and older leaves make the location of tea nutrient buds challenging. This research presents a lightweight and efficient detection model, T-YOLO, for the accurate detection of tea nutrient buds in unstructured environments. First, a lightweight module, C2fG2, and an efficient feature extraction module, DBS, are introduced into the backbone and neck of the YOLOv5 baseline model. Second, the head network of the model is pruned to achieve further lightweighting. Finally, the dynamic detection head is integrated to mitigate the feature loss caused by lightweighting. The experimental data show that T-YOLO achieves a mean average precision (mAP) of 84.1%, the total number of parameters for model training (Params) is 11.26 million (M), and the number of floating-point operations (FLOPs) is 17.2 Giga (G). Compared with the baseline YOLOv5 model, T-YOLO reduces Params by 47% and lowers FLOPs by 65%. T-YOLO also outperforms the existing optimal detection YOLOv8 model by 7.5% in terms of mAP. The T-YOLO model proposed in this study performs well in detecting small tea nutrient buds. It provides a decision-making basis for tea farmers to manage smart tea gardens. The T-YOLO model outperforms mainstream detection models on the public dataset, Global Wheat Head Detection (GWHD), which offers a reference for the construction of lightweight and efficient detection models for other small target crops. © 2024 Society of Chemical Industry.

BotanicGarden: A High-Quality Dataset for Robot Navigation in Unstructured Natural Environments

BotanicGarden: A High-Quality Dataset for Robot Navigation in Unstructured Natural Environments